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ABSTRACT 


Stauffer, Griffin, K. MSCE, Purdue University, August, 2006. 

Design-Build vs. Dcsign-Bid-Buitd: A Procurement Method Selection Framework. 
Major Professor: Luh-Mann Chang. 


Proper procurement method selection is an integral part of project success. Better 
informed owners arc able to more successfully select the project delivery systems that 
best suit their needs. This study utilizes utility theory to construct a framework to assist 
in the procurement decision making process. Through the use of expert weighting of 
important procurement criteria, real world projects were used to develop an overall 
threshold to which future owner’s can compare their subsequent projects. This threshold, 
which marks the boundary between Design-Build and Design-Bid-Build, can be used to 
measure an owner’s propensity to use cither procurement method. It is fully taiiorablc to 
any owner, as owner-specific inputs arc used, 

This ability for owners to objectify the largely subjective procurement decision 
making process allows owners to create a predictable, measurable trend, thereby 
improving their overall decision making ability. 


1 


CHAPTER!. INTRODUCTION 

1.1 Problem Statement 

A rcoccurring question facing owners is how to best meet their evolving needs. 
When it comes to expanding mission or capacity, those needs arc often met through the 
construction of a new facility. One important driving factor contributing to the success of 
the new facility is the procurement method used to deliver the facility to the owner. The 
two most prevalent procurement methods, Design-Build (DB) and Design-Bid-Build 
(DBB), are well known. Less available, however, arc tailorablc tools or guidelines that 
illustrate or suggest when a particular procurement method is superior. It is perceived 
that more research is needed to develop a useful tool to assist in procurement method 
decision making. 


1.2 Objectives 

Procurement method selection is directly linked to owners' project goals, 
preferences, and unique circumstances surrounding their specific projects [ 1 ]. The goal 
of this research is to explore a generic framework to assist owners in determining which 
procurement method, specifically Design-Build or Dcsign-Bid-Build, would be most 
beneficial to meeting their unique needs. 

Several factors play a role in the level of success of a construction project. 
Different procurement methods each possess unique characteristics, emphasize different 
focus areas, and have inherent strengths and weaknesses [2], Owners’ failure to identify 
or understand these characteristics place them in a precarious position when committing 
funds to a project. At best, they run the risk of not giving themselves the fullest possible 
chance to fulfill their expectations, and at worst, may position themselves where different 
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aspects of the project and/or project team goals are in direct competition, possibly 
resulting in project failure [5,16], 

In an attempt to improve the owner’s ability to make informed procurement 
method decisions, it is hypothesized that by following the procurement selection 
framework constructed herein to determine the procurement method best suited to a 
particular owner’s needs, an owner’s requirements will be better satisfied than if an 
alternate method is used. The model incorporates several expert opinions and transforms 
them into a singular recommendation through the use of utility theory. This compilation 
of expert input into the framework of procurement method decision making delivers 
proven expertise into the hands of the owner. 

The supporting evidence behind the procurement selection model rests largely on 
expert input throughout the entire process. As outlined by Love et ai, given the high 
variability of application across numerous projects, objective measures cannot be used to 
obtain procurement method selection criteria [3,5]. Therefore, actual project data will be 
used to validate the model, and experts will evaluate the actual degree to which their 
expectations and needs were met [3]. Similarly, expert surveys have proven highly 
useful in determining construction industry performance and productivity trends [4], 

Results of this study arc generic and can be adapted to other owners utilizing 
construction as a means to meet their expanding needs. This study utilized U.S. Navy 
data to exemplify the model. For the U.S. Navy, making better procurement selection 
decisions carries with it a substantial monetary savings. Since the Navy’s construction 
organization, the U. S. Naval Facilities Engineering Command (NAVFAC), is a global 
entity performing over $ 10 billion in business annually, even slight improvement in 
selecting how to procure facilities to meet its needs can translate into significant savings. 
In short, this framework can help to further the Navy’s goal of establishing “best 
practices” for the government as well as being fully adaptable to any owner desiring 
improved procurement method decision making. 

Additionally, the need for the U.S. Navy to strive for the best value in 
procurement goes beyond simple economics and improved profitability. Though 
profitability might be the ease in the civilian sector, the requirement for a government to 
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best utilize its taxpayer’s dollars adds an important social context to the equation, The 
socio-political impacts of proper public funds expenditures can be significant and should 
in no case be overlooked. Proper spending in the form of best value projects resulting 
from proper procurement is a guiding principle that the government must adhere to, as it 
will certainly be judged by it. 


1.3 Organization 

This report outlines the methodology and process used in reaching an overall 
procurement method threshold between DB and DBB. It presents a literature background 
and discusses the different characteristics of each procurement method. It also offers a 
synopsis of previous work performed in this field and its influence on this report. The 
data collected from the experts is presented and subsequently analyzed; examples are 
provided to demonstrate the various formulas used in the calculation of each step. 

Finally, overall findings arc presented and discussed, along with recommendations, 
possible future extensions of this work, and overall conclusions. 

1.4 Research Summary 

Through expert input on important procurement method selection criteria and 
application to real-world projects, this research demonstrates a framework that assists in 
improving future procurement method decision making. Utility functions synthesized the 
raw data to yield repeatable, predictable results and the formulation of an overall 
threshold. Owners arc able to apply their future projects to this framework in a unique 
comparison of their current situation to the reference data. It is expected that those 
projects scoring above the threshold lend themselves to being more successful if procured 
as DB. Similarly, projects scoring below the model’s threshold are expected to be belter 
suited to be delivered via DBB means. 
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CHAPTER 2. RESEARCH METHODOLOGY 

2.1 Methodology 

The crux of the model lies in the selection and prioritization of certain 
procurement selection criteria to consider when choosing a procurement method, Owners 
must determine their position on each of these criteria prior to making a procurement 
method selection. Examples of these criteria include sustainability and speed required to 
deliver a facility. 

An initial list of criteria was developed through literature review and content 
analysis. Experts were asked to evaluate the criteria on their level of importance for 
consideration prior to deciding on a particular procurement method. If experts felt there 
was an important criterion absent from the list provided to them, they were asked to add 
it and score it accordingly. Next, the list was streamlined to include only those criteria 
that the group of experts collectively graded as the most important. Third, the experts 
were asked to assign relative weights to the criteria, evaluate their positions on real-world 
projects relative to each criterion, rate their overall level of satisfaction with the project, 
and offer any confounding factors present. Finally, utility theory was used to convert the 
experts’ marks into overall criterion scores. Utility theory was further utilized to compile 
these overall criterion scores together with their corresponding expert assigned weights 
into an overall threshold. This threshold, or total utility value, indicates the framework’s 
recommendations of either DB or DBB to increase owner satisfaction. 

This method allowed this independent research study to establish a reference to 
which future projects can be compared based upon how experts score individual 
procurement performance criteria. This can be extremely versatile and useful in future 
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projects as a way to evaluate an owner’s specific project situation. 

The independent variable used in this report was the individual project’s score for 
each criterion. This scoring of specific performance criteria reflected the unique result of 
the expert’s situation prior to the procurement of the project. This score proved a solid 
independent variable on which to determine relative dependent variable performance. 

The overall threshold, or dependant variable, is a product of a utility function and 
consequently serves as the reference to which an owner can compare future projects. 

As with any study, the model offered herein cannot completely capture all 
possible variables contributing to each project’s outcome. While procurement method 
selection plays a prominent role in determining project success, other confounding factors 
may be present that contribute (cither positively or negatively) to the results suggested by 
procurement method selection alone. 

Possible confounding factors may include poor contractor performance, unilateral 
funding reduction on the part of the owner, or other unforeseen circumstances. Any 
confounding factors encountered on a project will be noted by the experts. This will be 
accomplished by asking the experts in addition to scoring each project, to annotate if 
there were specific situations that affected project performance external to the 
procurement criteria in question. 


2.2 Research Process 

Literature review, expert input, and personal experience contributed to the 
creation of an initial procurement method performance criteria listing [5], This list 
contained the top 13 criterion thought to matter most in determining which procurement 
method will best meet an owner’s needs (see Table 2.1). In short, these criteria were 
thought to be the top parameters on which owners must define their position when going 
through the procurement method selection process. 

Once an initial criteria list was built, that list was sent to 5 experts on facility 
management and source selection. The experts were asked to use their professional 
judgment to independently rank which of the criteria they felt was most important to 
consider when making a procurement method selection. The results of the expert's 
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Criteria 

1 Availability of experienced contractors 

2 Flexibility of owner's requirements 

3 Need for owner involvement during process 

4 Owner's experience 

5 Owner's in-house technical capability 

6 Owner's willingness to accept risk 

j Presence of known site factors that may cause 
problems 

8 Project completion within original budget 

9 Project completion within original schedule 

10 Project size 

11 Project type 

12 Speed to deliver project 

13 Sustainability of structure _ 

Tabic 2.1: Criteria Listing 

rankings were collected and compiled into a single list. A 2-sample T test was then 
applied to the data to determine if any of the graded criteria were statistically less 
significant than the rest. Any criterion failing the T test for significance was discarded 
from further analysis. This yielded a final listing that included only the top criteria that 
the experts collectively chose as most important (see Figure 2.1). 



Figure 2.1: Criteria Listing Evolution and Project Scoring 


Once the final list of collectively chosen important decision influencing criteria 
was constructed, it was returned to the experts for weighting and use in the evaluation of 
real-world projects. Each expert was asked to provide a single overall weight for each 
criterion, summing to 100%. Additionally, each expert was asked to score 3 projects 
relative to each item on the list of criteria. For each criterion, a range from 0-10 was 
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developed representing the two opposite extremes of the criterion. For example, for the 
criterion ‘availability of experienced contractors’, the scale ranged from ‘very limited 
competitive environment’ to ‘several contractors available in market’. Projects 
represented both the DB and DBB procurement method. Experts placed a mark 
corresponding to their specific position for that project relative to each criterion. For 
example, a project to construct a barracks in time for a ship’s return to port may have 
caused an expert to score the criterion of ‘speed required to deliver facility’ on the 
extreme upper end of that criterion’s range, corresponding with ‘outside time constraints 
present and highly influential of project success’. 

In addition to scoring each criterion and annotating which procurement was 
actually used, the experts were also asked to comment generally on the extent to which 
the project satisfied their needs as well as any unique circumstances surrounding the 
project. These other circumstances were collected to help explain project success (or 
failure) outside of the procurement method. 

Once the project evaluations had been collected from each expert, results were 
collated to yield average highs and average lows for each criterion. These averages were 
used to develop corresponding utility functions [4]. Figure 2.2 illustrates a generic 
representation of a utility function generated in this study. All criterion scores above the 
average high value were fixed as totally Design-Build, and all scores below the average 
low value were fixed as totally Dcsign-Bid-Build. In other words, the example in Figure 
2.2 shows that any value above 4.5 is 100% suited for DB and any value below 1.5 is 0% 
suited for DB. Only in the range between the average low and average high is there 
variation in the utility score. This variation is determined by the equation of the function 
and ranges between 0 and I. (In this example, the variation between 0 and 1 is 
determined by the equation u(y) = 0.33y - 0.5.) This utility function, combined with the 
product of the criterion score and corresponding weight, yielded a total criterion utility 
value. 

The use of utility functions as a means to evaluate multiple criteria has been 
effectively utilized in the past [19]. According to Baird, the term ‘utility’ refers to the 
measurement of relative “liking” on the part of an owner for particular outcomes [20], A 
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major benefit to using this approach is that traditional multiple attribute utility theory 
provides a methodology for selecting from among a set of alternatives in the presence of 
uncertainty [21]. This allows several independent variables or decision outcomes to be 
described as probability density functions, thus uniformly representing varying inputs as 
a common output [19]. This common output serves as a singular representation of 
several different owner, project, and external environment characteristics. 



The summation of all criterion utility values resulted in a single project utility 
value. Averaging all project utility values yielded an overall total utility value, or 
framework threshold. This threshold, based directly on the expert-inputted performance 
criteria scores, serves as a reference, or boundary, between the two procurement types, 
DB and DBB. Future projects can be compared to this total utility value to assist in 
procurement selection; higher values lend themselves to DB, while values lower than the 
threshold arc better suited for DBB. 
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CHAPTER 3. PROCUREMENT SELECTION PARAMETERS 

3.1 Literature Review 

A brief overview, history, and trend analysis of the Design-Build and Design-Bid- 
Build procurement methods follow in an attempt to describe the choices that owners face 
for project delivery system. Figure 3.1 illustrates the different owner-contractor-designer 
relationships present in each delivery method. 

Under Dcsign-Bid-Build, the owner enlists an architect to prepare the design of 
the complete facility, including construction drawings, specifications, and contract 
packages. The design package is then presented to general contractors who bid for the 
work and engage subcontractors to provide various aspects of the project. Usually the 
lowest bid is selected. The selected contractor is then responsible for building the facility 
according to the design. With this project delivery system, the owner retains increased 
control over the project due to the separate selection of architect and contractor [6], 



Figure 3.1: Procurement Method Relationships 
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In contrast, when Design-Build is the procurement method of choice, an owner 
contracts with a DB team, which is often a joint venture of a general contractor and a 
designer. Usually led by a general contractor (though a push for an increase in 
architcct/dcsigncr lead teams is in progress [7]), the team designs the facility based on 
discussions with the owner about the needed functionality of the new facility. At an early 
point in the process, the DB team and the owner negotiate a contract to complete the 
design and construction of the facility. Once the owner approves the design, the design- 
build team is then responsible for construction of the project and for the coordination 
between design and construction [6], 

Though often referred to as an 'alternative delivery method', Design-Build is not 
a new concept. In fact, it is most likely the earliest and oldest form of project delivery. 

Its roots originate in the ancient "Master Builder" concept where responsibility for both 
design and construction resided with one person [8]. From the Great Pyramids to the 
Parthenon, owners looked to a single person to both envision a stmeture that met the 
owner’s needs and construct that structure based on a design that took specific 
constructability-related issues into account. 

During the last century, however, project procurement systems have primarily 
utilized the so-called “traditional” process of Design-Bid-Build [9]. The federal 
government has undoubtedly steered the public sector toward DBB through the 
enactment of federal laws such the Federal Acquisition Regulation (FAR) and the 1972 
Brooks Act [10], As originally passed, these statues inhibited the use of DB in public 
projects, and state and local procurement statutes generally followed the federal 
procurement models. As a result of the continuous use of these approaches, a litigious 
environment of clients, contractors, lawyers, and claim consultants since the 1970’s had 
begun to characterize the industry [10]. By the 1990’s, however, issues such as 
accelerated project delivery, qualification of bidders, lack of innovation, and quality 
construction had challenged the public sector to rethink its position on other alternative 
delivery methods and restructure governing regulations to allow the use of Design-BuiId 
[10,13]. 



Figure 3*2: Current DB Regulatory Environment [11] 


This amending of regulations has resulted in the current project delivery market 
experiencing a resurgence in the use of Design-Bui Id procurement. Figure 3.2 illustrates 
the current U.S. regulatory environment as it relates to Design-Build [11]. Figure 3.3 
outlines the historical usage of both DB and DBB and provides the Design-Build institute 
of America’s future projections on their use [11]. 


Non-Residential Design and Construction 
in the United States 



Figure 3.3: Procurement Method Usage Trends (11) 



































12 


3.2 Procurement Method Characteristics 
As stated earlier, different procurement methods each possess unique 
characteristics, which lead to advantages and disadvantages of using each one. 

According to Ling, et al, it has been shown that most clients consistently use the 
procurement method with which they arc most familiar [14]. Unfortunately, relatively 
few construction industry professionals both fully understand the differences in the 
various procurement systems and are subsequently unable to make recommendations on 
which system would be more appropriate for a specific project. It is therefore no surprise 
why owners and building professionals often resort to the procurement system with 
which they arc most familiar, regardless of its appropriateness for the project [14]. 

Indeed, the DBB method has survived for so many years because it has several 
advantages. These include familiarity to participants in the construction process, firmly 
defined contractual relationships, and increased owner control over the design through 
direct contractual relationship with the designer [14]. 

DBB docs, however, contain some limitations. Most notably is the greater 
propensity for conflict due to greater fragmentation throughout the project. In Design- 
Bid-Build, the designer creates the plans for a structure with little to no outside influence. 
The constructor then bids upon these documents on which he has had no constructive 
input. In fact, design and constructor rarely interact during the construction phase unless 
there is a conflict. Sitting squarely in the middle of this conflict (given his separate 
contractual relationships) is the owner. As a result, the contractor is encouraged to 
submit for extra payment to correct any design deficiencies (which takes added time), and 
the designer is encouraged to report that any deficiencies arc due to improper 
construction by the contractor. This adversarial situation often results in the owner 
bearing increased cost and/or forcibly accepting increased time to complete the project. 

The owner’s single relationship and point of responsibility to the contractor is said 
to be the greatest strength of Design-Build [14]. Owners arc able to enjoy efficient and 
centralized accountability and authority. In DB, there is no adversarial relationship 
between contractor and designer since they work together as a single project team [14]. 
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Both the ability for this combined team to resolve issues internally much faster than in 
DBB along with the often non-linear project schedule often results in reduced time 
necessary to complete the project. 

DB has its disadvantages as well. Because of its ability for design and 
construction to take place non-lincarly, an owner must commit to decisions earlier in the 
process. While not an absolute necessity in DB, “fast-tracking”, or the construction of 
certain approved portions of the project prior to a complete set of project plan documents, 
is not uncommon. As in any construction project, the changing of a parameter by the 
owner once construction has begun is much more costly than had the owner changed it 
prior to construction. 

Table 3.1 outlines some additional strengths and weaknesses of each procurement 
method for quick reference. 


Procurement 

Method 

Strength 

Weakness 

Design- 

Build 

Team concept focuses on common goals 
and objectives as single responsible entity 
for entire project 

Requires decisions to be made earlier in 
the process 

Takes advantage of the contractor's 
ingenuity and innovation 

More difficult to question decisions of 
engineer or architect of record 

Saves time because it is on a faster track 
with a faster schedule delivery 

Commitment, communication, and trust, 
must be implemented early in the process 

Enhanced constructability infused into 
design 

Owner ignorance to possible overdesign 

Design-Bid- 

Build 

Design is more precise and detailed at 
outset 

More likely to be a higher overall cost 
and require a longer schedule 

The engineer/architect of record works for 
the owner 

Increased probability of disputes 


Table 3.1: Procurement Method Characteristics )2]. 


3.3 Procurement Method Selection Research 
As a result of studies conducted in the past, a consensus exists that while one 
procurement method may outperform another for a specific project, no single 
procurement selection method is superior to others for any project [3], Furthermore, Luu 
et al. points out that a mutually exclusive list of procurement selection parameters hardly 
exists due to the uniqueness of the myriad of construction projects today [5]. Therefore, 
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prior research in this area demonstrates that a tailored approach involving surveys of 
procurement experts is the preferred method by which to collect data [3,5,9]. 

As this project’s focus is toward the U. S. Navy, the project experts polled in this 
study were from the Naval Facilities Engineering Command. NAVFAC is responsible to 
the U. S. Navy for over $10 billion in annual work. In a hierarchical system typical of 
the U. S. Government, NAFVAC is divided into three levels: an overall headquarters, 
two overall regional commands (NAVFAC Atlantic and Pacific), 12 geographic divisions 
(NAVFAC Northwest, Southwest, Europe, Hawaii, etc.), and several Integrated Project 
Teams (IPTs) consisting of contract field offices and installation public works offices. 
Appendix A has a brief graphical breakdown ol'lhe NAVFAC organization. The experts 
in this study are employed at the 1PT level. They represent the level at which most 
project-level decisions arc made. They routinely decide which procurement method to 
utilize on projects. On average, the experts in this study have over 18 years of experience 
in procurement selection; ail have a minimum of 10 years of experience. These experts 
serve as cither project managers, business directors, or public works officers responsible 
for overall management of construction and maintenance on a naval installation. 

Research into procurement methods and the related success or failure of each 
method has been accomplished in the past. Among the most notable arc Kumaraswamy 
and Dissanayaka [16,17], Molcnaar [12], Love, Skitmorc, and Earl [3], and Konchar and 
Sanvido [ 13]. Kumaraswamy and Dissanayaka explored the impacts of both procurement 
and non-procurement related variables (such as project conditions and team 
characteristics) on project performance by measuring 11 performance areas [16, 17], 
Love, Skitmorc, and Earl used expert surveys to poll attitudes toward different delivery 
options [3], while Molcnaar specifically investigated the DB delivery system and 
developed key criteria to measure performance in comparison to other project delivery 
systems such as DBB [12], Similarly, Konchar and Sanvido relied on empirical studies 
to develop specific Design-Build performance parameters upon which they could 
recommend the use of DB [ 13], 

Luu, et o/.’s work took elements of many of these previous studies into account 
when developing a listing of procurement selection parameters to be considered prior to 
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procurement selection. Luu, et al. also utilized expert input to determine specific areas 
of interest (client’s characteristics and objectives, project characteristics, and external 
environment) while developing a model to evaluate procurement selection methods [5], 
The parameters outlined by Luu, et al. (criteria 1-2, and 4-11) were selected following 
responses by 84 industry leaders as to what items were most important to consider prior 
to procurement method selection. This independent research study draws on several of 
the above listed individual’s work for support. 

While Luu, et al. provided a solid basis for the initial criteria listing generated in 
this study, the initial listing also included Hibberd and Basden’s important criteria of 
"owner’s propensity to accept risk” [15]. This facet gave insight into an owner’s 
propensity to assume a decision-making role throughout the project’s life. Criteria 3, 12, 
and 13 were also added to the initial listing. They were not explicitly annotated in prior 
research, but were considered important through the investigator’s field experience. 
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CHAPTER 4. DATA COLLECTION 

4.1 Procurement Selection Criteria List Formulation 
Table 4.1 shows the initial criteria listing sent for expert review. The 
experts were asked to score the importance of each criteria listed as well as add and score 
any additional criteria they felt demanded consideration. 

The aim of this initial listing was to adequately represent the various main 
categories of inputs into procurement selection: owner characteristics, project 
characteristics, and external environment [5]. To that end, criteria such as “need for 
owner involvement during process” and “owner's willingness to accept risk” represent 
owner characteristics. Similarly, criteria such as “project size” and “project type” 
illustrate project characteristics, and “availability of experienced contractors” and 
“presence of known site factors that may cause problems” represent external environment 


factors. 
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Type 

Criteria name 

Further criteria description 


Flexibility of o wner's requirements 

Very narrow specification allowing for little or 
no contractor flexibility vs. widely flexible 
performance specification 

U 

Vi 

■g 

O 

I 

j 

CJ 

Need for owner involvement during process 

Extent to which owner must be involved in 
design review, specification creation or 
enforcement, etc. 

Owner's experience 

Owner's experience with similar projects 

Am 

0 wnc r's i n-house tec h n i ea 1 cap ability 

Owner's ability to act as design engineer or 
answer technical questions during construction 


Owner’s willingness to accept risk 

Propensity for owner to assume decision 
making role throughout project design & 
construction 

13 

E > 

Presence of known site factors that may cause 
problems 

Soil conditions, unknown utility conditions, 
etc. 

m £ 
m 

A vailability of experienced contractors 

Extent of owner's pool of contractors from 
which to receive offers 


Project completion within original budget 

N/A 

tSi 

Project completion within original schedule 

N/A 

•1 

S3 

JC 

O 

Project size 

Owner's ability to manage contract via 
singlc/multiple contract s ) 

Project type 

|New technology or innovative vs. well 
understood or proven scope 

u 

$ 

Speed to deliver project 

Necessity to meet delivery date due to outside 
constraint vs. ability for schedule to slip 


Sustainability of structure 

Whether sustainable or other "green 11 building 
techniques arc to be implemented 


Tabic 4.1: Initial Criteria Listing with Descriptions 


Experts receiving the initial criteria listing were asked u> rate the level of each 
criterion’s importance using a 6-point Likert scale. Likert scaling is widely used in 
measuring respondents’ replies in questionnaires and is a demonstrated acceptable 
method in procurement method research [5], Responses ranged from criteria being 
extremely important and receiving a score of 5 to not important at all and receiving a 
score ofO. Table 4.2 illustrates the range of possible responses and amplifying 
descriptions of consideration frequency. 
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Score 

Response 

Description 

5 

Extremely Important 

Owners should consider this criterion every time 

4 

Very Important 

Owners should consider this criterion most times 

3 

Somewhat Important 

Owners should consider this criterion occasionally 
or only on specific projects 

2 

Somewhat Unimportant 

Owners may or may not consider this criterion 
without expected cITcct 

1 

Not Very Important 

Owners seldom consider this criterion or only for 
specifically unique projects 

0 

Not at All Important 

Owners rarely if ever consider this criterion 


Tabic 4 . 2 : Criteria Listing Response Scale 


The results of each expert’s responses were collected and compiled into Tabic 4,3. 
Averages and standard deviations were calculated for each. These were later used to 
determine if any criteria should be discarded from further analysis. 


Criterion 

Expert 1 

Expert 2 

Expert 3 

Expert 4 

Expert 5 

Average 

Std Dev 

1 Availability of experienced contractors 

1 

5 

6 

4 

4 

3.80 

1.64 

2 Flexibility of owner’s requirements 

3 

3 

4 

4 

4 

3.60 

0.55 

3 Need for owner involvement during 

4 

2 

2 

5 

5 

3.60 

152 

process 








4 Owner's experience 

1 

2 

5 

3 

3 

2.80 

148 

5 Owner's in-house technical capability 

3 

4 

4 

3 

5 

3.80 

0.84 

6 Owner's willingness to accept risk 

5 

3 

5 

3 

4 

4.00 

100 

7 Presence of known site factors that 

1 

4 

5 

4 

5 

3.80 

1.64 

may cause problems 





_ Project completion within original 

o . 

4 

4 

5 

5 

5 

4.60 

0.55 

budget 






g Project completion within original 

5 

4 

5 

4 

4 

4.40 

0.55 

schedule 






10 Project size 

3 

2 

3 

4 

3 

3.00 

0.71 

11 Project type 

1 

3 

3 

4 

5 

3.20 

148 

12 Speed to deliver project 

5 

3 

5 

4 

4 

4.20 

0.84 

13 Sustainability of structure 

2 

2 

4 

4 

5 

3.40 

1.34 


Table 4.3: Criteria List Expert Responses 


4.2 Project Scoring and Expert Evaluation 
The second step in this survey asked the experts to score 3 projects from their 
installations on each of the criteria. Successful responses were received from all experts 
queried and the projects received represented a range of sizes and both Design-Bui Id and 
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Dcsign-Bid-Build project delivery systems. See Figure 4.1 for a graphical 
representation of the projects characteristics. 



Figure 4.1: Study Project Characteristics 


These projects served as the building blocks of the procurement framework. The 
scores that the experts submitted were the inputs that determined the corresponding utility 
function shapes and ultimately led to an overall utility value. Appendices G through K 
contain the scoring data for each expert. 

In addition to scoring individual projects, experts also provided weights for each 
criterion. These weights represented the relative importance each criterion had with 
respect to one another Table 4.4 shows the compiled result of each expert's input and 
the average value used in the further calculations of utility functions. 


Criterion 

Expert 1 

Expert 2 

Expert 3 

Expert 4 

Expert 5 Average Std Dev 

1 

5% 

30% 

13% 

5% 

10% 

12.6% 

0.103 

2 

5% 

5% 

6% 

0% 

5% 

4.2% 

0.024 

3 

10% 

5% 

5% 

0% 

10% 

6.0% 

0.042 

4 

1% 

5% 

10% 

10% 

5% 

6,2% 

0.038 

5 

2% 

5% 

6% 

10% 

10% 

6.6% 

0.034 

6 

5% 

5% 

10% 

0% 

5% 

5.0% 

0.035 

7 

10% 

5% 

5% 

0% 

5% 

5.0% 

0.035 

8 

5% 

10% 

12% 

30% 

15% 

14.4% 

0.094 

9 

15% 

10% 

12% 

15% 

5% 

11.4% 

0.042 

10 

20% 

5% 

2% 

5% 

5% 

7.4% 

0.072 

11 

5% 

5% 

2% 

0% 

0% 

2.4% 

0.025 

12 

15% 

10% 

7% 

5% 

5% 

8.4% 

0.042 

13 

2% 

0% 

10% 

20% 

20% 

10.4% 

0.095 


Table 4.4: Expert criteria weights 












20 


CHAPTER 5. DATA ANALYSIS 


5.1 Criteria List Analysis 


Once the experts’ results were received on the initial criteria listing survey, it was 
necessary to analyze them to determine if any of the criteria ought to be excluded from 
further calculations due to insignificance with the remainder of the data. This was done 
utilizing 2-samplc T test methodology. Figure 5.1 illustrates the governing formula for 

the t-vaiue used in the 2-samplc test where x is the average criterion score, s is the 
standard deviation, and n is the sample size [18]. 



Figure 5.1; l-value equation 


Rearranging the equation in Figure 5.1 solves for the difference in two criterion’s 
means (See Figure 5.2). 



Figure 5.2: Significant mean difference equation 


For example, the t-value corresponding to a 90% degree of confidence and sample 
size of 10 is 1.38 [18]. With standard deviations of 1.5 and 1.8 respectively, the equation 
in Figure 5.2 yields the result illustrated in Figure 5.3. 
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_ _ |t 1 o 2 

x.-x, =1.38* j—+— =1.02 
1 ‘ V 10 10 

Figure 5,3: Example significant mean difference calculation 

Therefore, given the confidence level and sample size, the result is that for xt to be 
significantly different than xi, its mean must differ from xi 's by more than 1.02. 

For this study, a confidence level of 75% was utilized. This was chosen because 
the inputs arc, unlike highly predictable laboratory experiments, based on human 
experience and therefore widely variable. Using a 90% or higher level of confidence was 
therefore avoided, and lower confidence values were not chosen to retain mcaningfulncss 
in the resulting analysis. The t-value corresponding to a 75% degree of confidence and a 
sample size of 5 is 0.741 [18]. 

The result in this study was for a criterion not to be considered significant when 
compared to the remainder of the group, its mean must have differed by at least 0.544 
based on the equation in Figure 5.2. Figure 5.4 shows the sorted list of criteria based on 
criteria averages. It also shows +/- one standard deviation for each criterion. “Project 
completion on original budget” received the highest average score of 4.6 while “owner’s 
experience” received the lowest at 2.8. 


6.00 
® e. no 

Criteria Score Ranges 
+/. 1 Std Dev 







o 

o a on 

t + i < 

> 1 







to 4 00 

a a™ 
i 2 00 

> 1 Art 

T T 

t ‘ 

► i 

F I • 

► , 

* i 

1 








t < 

* 




^ 1 .uu 

0.00 


Grig budget 

Orig sched 

Speed 

Risk 

Tech 

capability 

KIR 

availability 

Site factors 

Flexibility 

involvement 

Sustainability 

Proj type 

Pro) size 

Experience 


Figure 5,4: Initial criteria list results 
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All thirteen criteria’s means differed by 0.2 or less. Since none of the criteria 
scores were greater than 0.544 apart, it was determined that all thirteen criteria annotated 
on the initial list were significant and were therefore all included in further analysis. 
Additionally, nearly alt criteria yielded average values above 3.0, corresponding to 
“somewhat important”. Only ‘owner’s experience’ fell below that mark at 2.8, but was 
considered close enough to warrant continued inclusion. In essence, since none of the 
criteria needed to be discarded, the initial criteria listing became the final criteria listing. 

5.2 Utility Function Generation 

This framework applied procedures previously outlined in Chang, et al. utilizing 
utility functions to quantify subjective expert input [4], Since all 13 criteria were deemed 
valid for further consideration, this system requested that all 5 experts each score 3 
projects on all 13 criteria. The result was a database of fifteen projects representing a 
range of project types, sizes, and both DB and DBB delivery methods. The scores for 
these projects were used to calculate a utility function for each criterion. 

Earlier, Figure 2.2 showed how the utility function for each criterion encapsulated 
both the average high and average low scores received from the experts. For example, 
Table 5.1 demonstrates the approach for the third criterion “owner involvement during 
process”. It uses inputs from 2 experts scoring 3 projects each. 


Owner involvement during process 


1 

2 

3 

4 

5 

Expert 1 

X 

X 


X 


Expert 2 


XX 



X 


Tabic 5.1: Sample Expert scoring input 


From this data, average low and average high values were calculated. In this 
example, low values of 1 from Expert 1 and 2 from Expert 2 yield an average low value 
of 1.5. Similarly, high values of 4 from Expert 1 and 5 from Expert 2 yield an average 
high value of 4,5. Consequently, any value below the average low is fixed (in this 
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framework as Dcsign-Bid-Build), as is any value above the average high value (in this 
framework as Design-Build). Calculating the slope between the two points yields the 
equation of the line. In this example, the line rises from 0 to ! over a run of 3 (4.5 - 1.5). 
Therefore, the slope is calculated by dividing the rise by the run or a = 1/3 = 0.33. 
Substituting the Average Low (1.5, 0) into the equation for a line (u(y) = ay + b) and 
solving for b, the y-intercept is found to be 0 = (0.33 * 1.5) + b, therefore b - -0.5. 

Figure 5.5 illustrates this graphically. The framework repeats this process for each of the 
13 criteria, yielding 13 utility functions. 


Design-Burtd 


u(y) 


Design-Bid-Buiid 


Example Criterion 
Utility Function 



Figure 5.5: Sample Utility Function 


5.3 Project and Total Utility Value Generation 
Next, the average weight for each criterion, w, is used to indicate relative 
importance between criteria. Each expert assigned weights to each criterion. Those 
weights were averaged to produce an overall criterion weight. Figure 5.6 illustrates the 
results of the expert input, with “project completion within original budget” receiving the 
most weight of 14.4% and “project type” receiving the least relative weight of 2.4%. 
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Criteria Relative Weighting 



Owr>er Charactensties Project Characteristics External Env 


Figure 5.6: Criteria relative weighting results 


It is interesting to note that the criterion “availability of experienced contractors” 
received the second highest weight of 12.6%, yet did not break out from the group in 
survey #1 as very important. This can be explained by the averaging process. In an 
attempt to encompass the widest base of Navy experts for this study, experts from 
different parts of the country were asked to participate. One expert in particular was from 
Hawaii. That expert placed a relative weight of 30% on criterion, which was 
dramatically higher than other experts. This is due to the geographic isolation that 
Hawaii faces with respect to contractors available to perform work. The average value of 
12.6%, therefore, was a result of this expert’s 30% weighting; the average of the other 4 
experts’ weights was only 8.25%. It is expected that for most owners, this criterion 
would be weighted less than in this study, as most owners arc not faced with geographic 
situations typical of remote areas. 

The equation illustrated in Figure 5.7 demonstrates the method utilized to reach a 
utility value (U(y)) for each criterion [4], 


u i = w l *u i *y i 

Figure 5.7: Utility value equation 

Continuing the example on the third criterion “owner involvement in process,” 
individual expert weights of 0.01,0.05, 0.1, 0.1, and 0.05 were averaged to yield a weight 
for this criterion of 0.062. Assuming the next project evaluated was given a score of 3 
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for this criterion, the corresponding utility value would be 0.09114 as shown in Figure 
4.8 below. 


from utility Junction 

w = .062 u(y) = (.33 * 3) - .5 = .49 y = 3 

See Figure 5.5 

U 3 = w } *u J *y 1 —> .062*.49*3 = 0.09114 

Figure 5.8: Criterion U value sample calculation 

0.09114 is the utility value for the third criterion only, represented by Uj. To 
reach the project’s U value, this process must be repeated for each of the remaining 12 
criteria. The summation of all 13 criterion utility values (U[ through Uo) results in a 
single project U value (Uprojeci)- For example. Project 3 yielded criteria values of 0.711, 
0.752,0.056, -0.051,0.867, -0.021,0.138, 0.360,0.960, 0.056, 0.504,0.672, and -0.437 
for a project utility value of 4.57. This study yielded project U values ranging from -0.27 
to 13.28. Each project’s U values were then averaged to yield a total utility value, or 
threshold, for the entire study (Ut)- This threshold represents the collective input of the 
experts and reflects a single value to which future projects can be compared. Figure 5.9 
illustrates the hierarchical process undertaken to reach the total utility value [4], 


Total 

Criteria Project Vp . + * 7" ’ 

wy, =£/,-> 'Zu,...u„ = -> ^ = u. 

Figure 5.9: Hierarchical process used to reach Total Utility Value U T 

5.4 Purpose and Use of Threshold Value 
The framework described above yielded project utility values of 7,10. 9.76, 4.56, 
5.20, 3.04, 6.65, 9.09, 6.78, 6.98, 6.82, 6.23, 13.28, -0.27, -0.07, and 0.15 for each of the 
15 projects. Following the framework, the average of these project utility values yields a 
total utility value of 5.69. This number, therefore, represents these 5 Navy experts’ 
current position with respect to importance of procurement method selection criteria. 
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their relationship to one another in the form of relative weights, and their personal 
tolerance within a range of each criterion. Another way this can be expressed is to treat 
the threshold value as a “Navy industry average”. This is useful because future owners 
can utilize this same framework to establish their own average. As more projects arc 
undertaken, the total utility value will track as appropriate to reveal the owner’s true 
threshold between Design-Build and Dcsign-Bid-Buiid. 

For example, the current Navy threshold (understanding that “the Navy” is being 
represented by the 5 experts and 15 projects polled in this study) is 5.69 as stated earlier. 
As future projects come about, Navy owners can score them as they see fit (assigning 
individual y values to each of the 13 criteria), and the system will produce that project’s 
corresponding U value. This project U value will either be above or below the Navy 
threshold. If the project is substantially below the threshold, then it is well suited for 
Dcsign-Bid-Build. On the other hand, if the project is well above the Navy total utility 
value, it would serve the owner more to procure it via Design-Build, It is important to 
note that this is a sliding scale, meaning that the further away from the threshold a future 
project scores (either above or below) the more convincing it is to follow the procurement 
method recommended. 


5.5 Framework Recommendation vs. Actual Project Procurement Method 

Figure 5.10 illustrates framework performance in recommending procurement 
method. The threshold of 5.69 is clearly indicated by a solid green line. The clear area 



Figure 5.10: Actual vs. Recoin mended Procurement 
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surrounding the threshold is bound by +/- half of one standard deviation (a/2). Beyond 
this clear area arc red and blue regions corresponding to stronger recommendations of 
DB and DBB, respectively. Projects falling in the clear area arc close to the threshold 
value and can be reasonably expected to be successful if procured by either method. 

Overall, Projects 1,5-7, 9, 11, and 12 -15, or 67% of the projects were actually 
procured by the same method recommended by the framework (i.c. had project utility 
values above or below the threshold corresponding to cither DB or DBB). Specifically, 
75% of DB projects (Projects I, 6, 7,9, 11, and 12) agreed with the framework while 
57% of DBB projects (Projects 5, and 13-15) also followed the recommendation. While 
having two thirds overall support the framework lends credibility to the process, further 
investigation reveals an even stronger result. In the areas beyond half of one standard 
deviation, 85.7%, (Projects 5, 7, and 12-15) followed the framework. These are the areas 
which the framework more strongly recommends a particular procurement method, as it 
is further away from the midpoint where cither method may be acceptable. 

Nearly every project was either successful in its procurement method or poor 
performance explained through one or more confounding factors. Specifically, the 
projects located in the stronger areas of recommendations that followed the framework's 
recommendation had acceptable performance. The owner of Project 12 indicated that 
despite a tight budget and timeline, the Design-Build project had both successful design 
and construction phases. Additionally, the owner of Projects 13, 14, and 15 did not 
indicate any negative effects suffered during these projects. See Appendicies G-K for 
expert comments on each project. 

Project 2 is the only project that did not follow the framework outside of the clear 
area. This project was actually procured using DBB means while the framework clearly 
recommends DB for a project having a utility value of 9.7. The expert overseeing this 
project reported serious problems with the performance of this project. Specifically, he 
cited schedule delays and conflicts over design flaws and the need for change orders. 

This project involved the renovation of an existing facility. Renovation projects 
arc often subject to numerous changes due to unforeseen conditions differing from what 
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was originally specified or thought to be true on design documents, The inflexibility 
of the DBB contract to compensate for this fact resulted in low owner satisfaction and 
overall poor contract performance. The fact that the one and only project that did not 
follow the framework’s recommendation suffered from poor performance even further 
bolsters the credibility of the system. 

It is important to note that the projects used to develop this framework were not 
selected retrospectively based on their compliance with the study. Conversely, the 
framework was established first, and the fifteen projects (which made up the entire 
project database) were subsequently subjected to the process and the results calculated. 

5.6 Confounding Factors 

it is important to mention that simply following the procurement method 
recommended as a result of following this framework docs not guarantee project success. 
There are several confounding factors external to procurement method that may 
significantly influence a project’s outcome. In this study, the majority of the projects 
illustrated the standard strengths and weaknesses of their procurement type. According 
to the project owners, most DB procured projects completed on time and were 
sufficiently flexible to cope with varying design and site conditions. Furthermore, the 
DBB procured projects often completed behind schedule, had greater communication 
difficulties, and in extreme eases, were fraught with claims overdesign omissions and 
differing site conditions. 

Some projects used in this study, however, had confounding factors override the 
standard procurement method parameters. For example, a boiler replacement project 
procured via DB suffered from design omission discovered following project closeout. 
Similarly, an armory renovation and decontamination project procured via DB suffered 
from poor contractor performance in the form of a delayed start and subsequent late 
finish. Finally, a DBB project to install a flightline fuel tank delivered a successful 
product on time with minimal changes. These exceptions demonstrate that the 
procurement method alone docs not automatically deliver a successful project. 
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CHAPTER 6. RECOMMENDATIONS AND CONCLUSIONS 

6.1 Results and Findings 

Using U.S, Navy data, a total utility value, or threshold, was determined to be 
5.69 based on 5 experts and 15 projects. While not intended to be all inclusive, this study 
demonstrated a generic framework by which owners can follow to determine their own 
threshold between Design-Build and Design-Bid-Build. As future projects come about, 
owners can compare the new projects scores to their threshold to determine which 
delivery method better suits their individual needs. 

It was found that the framework successfully recommended either the DB or DBB 
procurement method based on owner inputs. Within the boundaries of 5 experts and 15 
projects this study’s data set included, 67% of projects tested agreed with the system 
regardless of distance from the threshold. Beyond half of one standard deviation, the 
framework properly recommended 87% of the projects’ actual procurement methods. 

The sole project in this realm that did not follow the framework’s recommendation 
suffered from poor contract performance and low owner satisfaction. 

6.2 Research Limitations 

It is important to note that while this study has produced encouraging results, it 
docs have its limitations. Specifically, the study group utilized in this work consisted of 
only five experts to represent the U.S. Navy, an extremely diverse owner. Additionally, 
the project database consisted of ! 5 projects, only a minute fraction of the total projects 
procured by this owner. This framework attempt to demonstrate method over individual 
figures, as these figures will be different for each owner utilizing this methodology. 
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6.3 Recommendations 

It is the recommendation of this study for owners to utilize the generic framework 
developed herein to improve their procurement method selection and decision making 
capability. By establishing a threshold unique to them, owners can quantify their 
subjective biases on various procurement criteria. Furthermore, they can compare future 
projects to this threshold to better select the procurement method that leads to project 
success. 

This framework can be improved through future extensions that subject a great 
many more projects to this framework. By polling several owners spanning different 
backgrounds, areas, and expertise in the construction industry, a large number of projects 
inputted into this framework would yield an overall industry average for many different 
construction sectors. This extension would add an additional benefit to the framework, 
allowing owners to not only compare their future projects against their own threshold, but 
against a threshold representing the sector of construction in which their project lies. 

6.4 Research Conclusions 

This study was successful in using utility theory to produce a potential framework 
to measure owner propensity toward either the Design-Build or Dcsign-Bid-Build 
procurement method. Its use will allow owners to quantify an often subjective process to 
assist in procurement method decision making. It is fully tailorable to the owners’ needs 
and reflects owners’ individual preferences toward their specific projects. The end result 
is more informed owners who can better determine the proper procurement method best 
suited to meet their needs. 
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This survey is attempting to gamer data on those criteria which owners should consider when choosing a particular procurement method, specifically DesigrvBuikl or Design -Bio-Build The cmeha are not 
intended to be biased toward any particular method 
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This- survey cs attempting to gather data on mass criteria which owners, should consider when choosing a particular procurement method, spacificairy Design-Build or Design-Bid-Build. The cniena ere not 







































Please proceed to the second tab labeled "Project Scoring", 
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Procurement Survey #2 

There ere 2 tabs for this survey. Please be sure to complete both tabs 
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There are 2 tabs for this survey. Please be sure to complete both tabs. 
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Please proceed to the second tab labeled "Project Sconng". 
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Procurement Survey #2 

There a/e 2 tabs for tfws survey. P/ease be sure to complete both tabs. 
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